Search information managing for moving image contents

ABSTRACT

A search information analyzing unit ( 101 ) separates search information ( 15 ) into structured search information ( 152 ) representing a structure of moving image contents, semantic search information ( 153 ) representing characteristics in semantic context of the moving image contents, and signal search information ( 154 ) representing physical characteristics of signals of the moving image contents, and outputs the results. A structural search information disassembling unit ( 102 ) extracts and outputs, from the applied structural search information, information corresponding to at least one of the semantic search information and signal search information as structural information ( 50 ). A unit search information forming unit ( 103, 104 ) adds, to at least one of the applied semantic search information and signal search information, to form a small unit of search information ( 153 A,  154 A). Such a small unit of search information can be used as search information for the moving image contents, and hence, the amount of search information can be reduced.

TECHNICAL FIELD

The present invention relates to apparatus and method for managinginformation having a tree structure (hereinafter referred to as searchinformation) for searching for moving image contents, and to a searchapparatus executing a search process. More specifically, the presentinvention relates to search information managing apparatus and methodenabling efficient storage or transmission of search information, aswell as to a search apparatus executing a search process using thesearch information.

BACKGROUND ART

For moving image contents, every time point or every spatial point(image plane) of the moving image contents can be an object of search.Therefore, in order to execute an exact search, it is necessary toplayback the moving image contents as a whole and to check in detail.Such a method of searching, however, is not at all efficient, and whenthe number of moving image contents as the target of search increases,the method would be even less practical.

Various methods have been proposed as methods enabling a user to quicklyand efficiently search for desired moving image contents or a desiredportion of the moving image contents, in which search information (whichsearch information is also referred to as metadata for search) to be.added to corresponding moving image contents is prepared, a search isexecuted based on the search information, and the desired moving imagecontents or the desired portion of the moving image contents isobtained.

FIG. 10 shows a schematic structure of general moving image contents.Such a structure is widely known and disclosed, for example, in JapanesePatent Laying-Open Nos. 10-108071 and 10-257436.

As can be seen from FIG. 10, the entire moving image is divided, alongtime, into a number of scenes. Here, the term scene refers to a sectionmainly of a semantic unity along the time axis, and scenes may includelarge sections such as part 1 and part 2 of a movie as well as a verysmall section such as a scene of “A and B fighting against each other.”Therefore, the actual division of the entire moving image into scenes isnot limited to one stage division such as shown in FIG. 13, but ratherproceeds gradually over a plurality of stages.

One scene is further divided into a number of shots (or into smallerscenes) along the time axis. Here, the term shot refers to one sectionalong time consisting of continuous frame images picked-up by oneoperation, and in the moving images, it refers to a section between twoediting points (cut points).

A shot is a kind of a scene, and the shot may further be divided intosmaller shots (sub-shots). Though not shown, the shot can eventually bedivided into one-by-one frame images.

In this manner, the moving image contents has a hierarchical structuredivided from the entire body to small portions and further to smallerportions. Corresponding search information is added to each level of thehierarchy.

FIG. 11 represents search information for the moving image contents ofFIG. 10. The search information of moving image contents is representedby a tree structure such as shown in FIG. 11. The search information ofmoving image contents includes structural search information, semanticsearch information and signal search information.

The structural search information refers to information representing thehierarchical structure of the moving image contents shown in FIG. 10.The structural search information includes structuring information forrepresenting the tree structure, that is, relation between variousscenes or shots of the entire moving images, and structural attributeinformation representing attribute of each scene or shot resulting fromthe division.

In the tree of FIG. 11, the root R represents the entire moving images,and each node ND represents each scene or shot obtained by the division.A node ID or level information appended to the root R or to each node NDconstitutes the structuring information, and a start time, end timeinformation or the like appended to each scene or shot constitutes thestructural attribute information.

The semantic search information refers to search information defined bysemantic characteristics of the moving image contents. For example, thetitle of the entire moving image contents, information of a producer,and information of visual scenic description of a scene constitute thesemantic search information.

The signal search information refers to search information defined byphysical characteristics of signals of the moving image contents. Forexample, color frequency information, motion intensity information orthe like for each shot (or for a representative frame image thereof)constitutes the signal search information. The semantic searchinformation and signal search information are provided as leaves RFlinked to each corresponding node ND.

FIG. 12 represents the manner of recording when the search informationhaving the tree structure shown in FIG. 11 is recorded. As can be seenfrom FIG. 12, the search information having the tree structure shown inFIG. 11 is recorded by tracing the root R and the nodes ND from shallowto deeper levels of the hierarchy (along the direction from the root Rto the nodes ND), and from left to right (earlier time point to latertime point), with priority given to the depth (when there is no node NDany longer to be traced in the lower level of one node ND, tracing isswitched to a neighboring node ND on the right side), and the searchinformation as the leaf RF added to the root R or the node ND isrecorded successively.

Though the highest level (root R) is shown as representing one, entiremoving image content in FIGS. 10 to 12, it is not limited to one movingimage content, and it may be considered as a set of a plurality ofmoving image contents. By way of example, the highest level may be videoprograms as a whole that are broadcast in one day on one channel. Thevideo programs as a whole broadcast in one day on one channel aredivided into programs one by one, that is, respective moving imagecontents, in the next level. Each of the moving image contents isdivided into a plurality of scenes, and each scene is divided into aplurality of shots.

By the search information having the tree structure, it becomes possibleto present the structure of the moving image contents along time in aform easily understandable by the user, associated with various piecesof search information. Therefore, the search information is well suitedfor operations such as editing of the corresponding moving imagecontents by the user with visual confirmation based on the searchinformation, or editing of the search information itself. It is noted,however, that when the size of the moving image contents becomes large,the size of the tree representing the search information also becomesvery large.

Assume an execution model in which moving image contents and searchinformation are stored in a server apparatus, a user accesses from aclient terminal to the server, obtains the search information andexecutes a search at the client terminal. Here, a situation is notdesirable in which a formidable amount of search information must befully obtained by the client terminal from the server before the usercan start operation of the client terminal to execute a search.Specifically, it is desired that the search information having the treestructure is divided into an appropriate plurality of small units ofsearch information and transmitted to the client terminal, and that theuser can execute a search to some extent at the client terminalutilizing the received small units of search information.

By dividing the search information having the tree structure inaccordance with sections along time axis into scenes or shotsconventionally, it is possible to form small units 11 to 14 of searchinformation shown in FIGS. 13A to 13D. The small units of searchinformation 11 to 14 can be obtained by dividing the search informationinto a set of a node ND and its leaves RF starting from each of thelowermost nodes ND of the tree, or a set of nodes ND and their leaves RFincluding a lowermost node and connected higher levels nodes ND, asshown in FIG. 14.

The small unit of search information obtained by division in accordancewith section along time holds the information of time structure of thecorresponding moving image contents as it is, and therefore, it can besuitably used for transmission in synchronization with the moving imagecontents, to enable filtering of the moving image contents on-line,based on the search information.

As disclosed in Japanese Patent Laying-Open No. 8-110912, it is wellknown to store the search information having the tree structure dividedinto the tree structure and a table of appended information representingattribute information of each of the nodes ND of the tree structure.According to this approach, there are two divisible units, that is, thetree structure and the table. The appended information table, however,cannot be handled independently, and the above described division alongtime section is not available.

In the present specification, the above described small unit of searchinformation obtained by the division will be referred to as “unit searchinformation.” Further, node ID information or the like for uniquelyidentifying a node ND of the structural search information will bereferred to as “search information element” of the structural searchinformation (or structural search information element), actiondescription information or the like of the semantic search informationwill be referred to as “search information element” of the semanticsearch information (or semantic search information element), and each ofcolor frequency information, motion intensity information and the likeof the signal search information will be referred to as “searchinformation element” of the signal search information (or signal searchinformation element).

The small unit of search information conventionally obtained by dividingthe search information having the tree structure is unit searchinformation corresponding to the time section, that is, the structuralelement of the moving image contents such as a scene or a shot.

In such unit search information, however, search information elements ofthe structural search information, semantic search information andsignal search information exist in a complicated manner. There may be,for example, a request that only the structural search information ofthe moving images is to be known (it is desired to know how the movingimages are divided into scenes or shots). Alternatively, there may be arequest that only the color frequency information of each shot isdesired to conduct a search based on color frequency information shot byshot, or a request that only the motion intensity information is desiredto conduct a search based on the motion intensity information shot byshot. Conventionally, a searcher must obtain the unit search informationthat includes the search information elements of the structural searchinformation, semantic search information and signal search informationexisting in a complicated manner, and the searcher must analyze and usethe obtained information to meet such requests.

It is efficient from the view point of a searcher that searchinformation elements of the same type (type such as the color frequencyinformation or motion intensity information) added to respective shotsor respective scenes are obtained collectively for a plurality of shotsor a plurality of scenes as the components of the moving images, andthat a search can be conducted by comparing and collating the same typeof information elements. For example, using the color frequencyinformation among the signal search information elements, a shot can besearched by detecting one that has color frequency information matchingto a desire among color frequency information added to the shots.

Though Japanese Patent Laying-Open No. 8-110912 contains a hint tohandle only the tree structure separate from the search information, itdoes not suggest handling of other appended information independently,or division and management based on detailed types of search informationsuch as described above.

The search information elements of the structural search information andthe signal search information are generally given in fixed formats. Incontrast, the search information elements of the semantic searchinformation often include free text data and the like of unfixed length,such as the data of action description.

Therefore, when the search information contains a plurality of searchinformation elements in a complicated manner, it is necessary to analyzeeach of the structural search information, signal search information andthe semantic search information even if only the search informationelements of the structural search information and the signal searchinformation of the fixed length are to be drawn out. This imposes aconsiderable burden on the searcher.

In contrast, when the search information is formed by collecting onlythe signal search information elements, the search information can behandled easily, for example, by skipping data of a fixed length.

As compared with the manner of search in which the search information isobtained containing the structural search information, semantic searchinformation and signal search information in a complicated manner inaccordance with time structure, the manner of search in which thestructural search information is obtained to enable understanding of thestructure of the moving images as a whole, and the signal searchinformation or the semantic search information is added as needed toproceed with the search is very natural. The conventional method ofdividing the search information, however, is not suitable for the lattermanner of searching.

The unit of search information conventionally obtained by division ofthe search information having the tree structure holds the informationof time structure of the corresponding moving image contents as it is,and therefore, it is suitable for a manner of search in which the searchinformation is transmitted in parallel with moving image contents, toenable filtering by an on-line search. When a large amount of data ofmoving image contents must be searched off-line, for example, and adesired shot is to be searched for simply by using specific searchinformation such as the color frequency information added to the entiremoving image contents, it has been necessary to read and analyze all thesearch information, in order to draw out the specific necessary searchinformation. Accordingly, there are many wasteful processes.

For an off-line search, it is natural to obtain only the searchinformation of the same type added to a plurality of moving imagecomponents, such as a plurality of shots, and to compare and collate thesame to conduct a search. By way of example, a server apparatus may drawout and transmit to a client terminal only the color frequencyinformation added to each of the plurality of shots, among the signalsearch information elements, the client terminal receives the colorfrequency information, and the user detects a desired shot based on thereceived color frequency information and requests the server apparatusonly the data of the corresponding shot.

As described above, the manner of division of the search information formoving image contents should be switched in accordance with the mannerof search. Conventionally, however, such a consideration has not beenmade.

DISCLOSURE OF THE INVENTION

An object of the present invention is to provide a search informationmanaging apparatus and a search information managing method that enableefficient use of search information having a tree structure added tomoving image contents, as well as to provide a search apparatus that iscapable of executing an efficient search process.

According to an aspect of the present invention, the search informationmanaging apparatus includes a separating unit, a structural informationextracting unit and a unit search information forming unit, for managingthe search information having the tree structure corresponding to themoving image contents divided into small units of search information.

The separating unit separates the search information into structuralsearch information representing the structure of the moving imagecontents, semantic search information representing characteristics insemantic context of the moving image contents, and signal searchinformation representing physical characteristics of signals of themoving image contents, and outputs the same. The structural informationextracting unit disassembles the structural search information outputfrom the separating unit, and from the disassembled structural searchinformation, extracts information that corresponds to at least one ofsemantic search information and signal search information. The unitsearch information forming unit forms a small unit of search informationby adding to at least one of the semantic search information and signalsearch information output from the separating unit the correspondingstructural information extracted by the structural informationextracting unit.

Therefore, a small unit of search information provided by adding to atleast one of the semantic search information and signal searchinformation separated from the search information the correspondingstructural information extracted by the structural informationextracting unit can be used as the search information for the movingimage contents, and hence, the amount of search information can bereduced. Accordingly, when a small unit of search information requestedby a searcher at the time of searching for moving image contents is tobe formed and transmitted to the source of request, the amount of searchinformation to be transmitted can be reduced.

According to another aspect of the present invention, the searchinformation managing apparatus includes a separating unit, a structuralinformation extracting unit and a unit search information forming unit,for managing the search information having the tree structurecorresponding to the moving image contents divided into small units ofsearch information.

The separating unit separates and draws out from the search information,one or a plurality of search information elements given by one or aplurality of leaves of the tree structure or given by one or a pluralityof sub-trees consisting of an arbitrary set of a node and leaves ornodes of lower levels linked to the node of the tree structure,representing elements of the same type included in the searchinformation. The structural information extracting unit extracts, fromthe search information, structural information corresponding to the oneor a plurality of search information elements drawn out by theseparating unit. The unit search information forming unit forms a smallunit of search information by arranging the one or a plurality of searchinformation elements drawn out by the separating unit and thecorresponding structural information extracted by the structuralinformation extracting unit in accordance with a prescribed format.

Thus, a small unit of search information is formed by arranging the oneor a plurality of search information elements drawn out from theoriginal search information and the corresponding structural informationin accordance with a prescribed format, and therefore, each of thesearch information elements included in the small unit of searchinformation can easily be specified. Accordingly, the small unit ofsearch information can easily be used at the time of executing a searchfor moving image contents.

In the above described search information managing apparatus, one or aplurality of search information elements drawn out by the separatingunit may correspond to the signal search information representingphysical characteristics of signals of the moving image contents.

As the signal search information representing physical characteristicsof signals of the moving image contents is used as the small unit ofsearch information, it becomes possible when, for example, the movingimage contents are to be searched using the color frequency informationonly, to use only the corresponding color frequency information fortransmission or search, and thus, an efficient search using only thesignal search information can be realized.

In the above described search information managing apparatus, one or aplurality of search information elements drawn out by the separatingunit may correspond to the semantic search information representingcharacteristics in semantic context of the moving image contents.

As the semantic search information representing characteristics insemantic context of the moving image contents is used as the small unitof search information, it becomes possible when, for example, the movingimage contents are to be searched using characters of each scene only,to use only the information of corresponding characters for transmissionor search. Thus, an efficient search using only the semantic searchinformation can be realized.

In the above described search information managing apparatus, theprescribed format may represent an arrangement of one or a plurality ofsearch information elements drawn out by the separating unit and thecorresponding structural information extracted by the structuralinformation extracting unit, in accordance with order of appearance ofthe one or a plurality of search information elements within the treestructure.

As the one or a plurality of search information elements and thecorresponding structural information constituting the small unit ofsearch information are arranged in accordance with the order ofappearance in the tree structure, information representingpreceding/succeeding relation of each of the search information elementscan be omitted. Accordingly, it becomes possible to reduce the signalamount of a small unit of search.

In the above described search information managing apparatus, the smallunit of search information may further be formed in the followingmanner. Specifically, the structural information extracting unitseparates the structural information into shared structural informationincluding type information shared by the one or a plurality of searchinformation elements drawn out by the separating unit or shared positioninformation of the tree structure, and un-shared structural informationother than the shared structural information. A small unit of searchinformation may be formed by arranging, in addition to the arrangementof the one or a plurality of search information elements drawn out bythe separating unit, the corresponding shared structural information andthe corresponding unshared structural information.

Therefore, the type information representing whether the searchinformation element represents color frequency or character information,or the position information representing at which level of the treestructure the search information element is connected, shared by thesearch information elements of the same type, is added to thearrangement of the one or a plurality of search information elements.Therefore, the type or the corresponding level of the tree structure caneasily be known for the search information element included in the smallunit of search information.

In accordance with a further aspect of the present invention, the searchinformation managing apparatus includes, for forming small units ofsearch information from search information having a tree structurecorresponding to moving image contents and for managing the same, arecord position information extracting unit, a structural informationextracting unit and a unit search information forming unit.

The record position information extracting unit extracts, for each ofone or a plurality of search information elements given by one or aplurality of leaves of the tree structure or given by one or a pluralityof sub-trees consisting of an arbitrary set of a node and leaves ornodes of lower levels linked to the node of the tree structure,representing elements of the same type included in the searchinformation, record position information representing the position wherethe search information element is recorded. The structural informationextracting unit extracts, from the search information, structuralinformation corresponding to the one or a plurality of searchinformation elements. The unit search information forming unit forms asmall unit of search information by arranging the record positioninformation extracted by the record position information extracting unitand the structural information extracted by the structural informationextracting unit corresponding to each of the one or a plurality ofsearch information elements, in accordance with a prescribed format.

Therefore, the one or a plurality of search information elementsrepresented by the small unit of search information can be specified bythe record position information arranged in the small unit of searchinformation, and the search information elements can easily be used atthe time of executing a search for the moving image contents. When asmall unit of search information is requested for searching for movingimage contents and the small unit of search information is transmittedto the source of request, it is possible for the source of request tospecify the desired search information element by the record positioninformation in the received small unit of search information without thenecessity of transmitting the search information element itself.Further, by transmitting only the desired search information element atevery request to the source of request, a highly efficient searchbecomes possible.

In the above described search information managing apparatus, the one ora plurality of search information elements may correspond to the signalsearch information representing physical characteristics of the signalson the moving image contents.

As the signal search information representing physical characteristicsof signals of the moving image contents is used as the searchinformation element corresponding to the record position information, itbecomes possible when, for example, a search is conducted using thecolor frequency information only, to use only the corresponding colorfrequency information for transmission or search, and thus, an efficientsearch using only the signal search information can be realized.

In the above described search information managing apparatus, the one ora plurality of search information elements may correspond to thesemantic search information representing characteristics in semanticcontext of the moving image contents. As the semantic search informationrepresenting characteristics in semantic context of the moving imagecontents can be used as the search information element corresponding tothe record position information, it becomes possible when, for example,a search is conducted using characters of each scene only, to use onlythe information of corresponding characters for transmission or search.Thus, an efficient search using only the semantic search information canbe realized.

In the above described search information managing apparatus, theprescribed format may represent an arrangement of respective recordposition information in accordance with the order of appearance in thetree structure of respective ones of the corresponding one or aplurality of search information elements.

Therefore, in the small unit of search information, informationrepresenting preceding/succeeding relation of each of the correspondingsearch information elements can be omitted. Accordingly, it becomespossible to reduce amount of information of the small unit of searchinformation.

In the above described search information managing apparatus, thestructural information extracting unit further separates the structuralinformation into shared structural information and un-shared structuralinformation other than the shared structural information. The sharedstructural information includes information representing type shared bythe one or a plurality of search information elements, or sharedposition information in the tree structure.

A small unit of search information may be formed by arranging, inaddition to the arrangement of the record position informationcorresponding to the one or a plurality of search information elements,the corresponding shared structural information and the correspondingunshared structural information, respectively.

Therefore, the type information representing whether the searchinformation element represents color frequency or character information,or the position information representing at which level of the treestructure the search information element is connected, shared by thesearch information elements of the same type, is arranged added to thearrangement of the record position information. Therefore, the type orthe corresponding level of the tree structure can easily be known forthe search information element represented by the record positioninformation included in the small unit of search information.

In accordance with a further aspect of the present invention, the searchinformation managing apparatus includes, for managing the searchinformation having a tree structure corresponding to moving imagecontents divided into small units of search information, first andsecond search information dividing units and a division selecting unit.

The first search information dividing unit divides the searchinformation into structural search information representing thestructure of the moving image contents, semantic search informationrepresenting characteristics in semantic context of the moving imagecontents, and signal search information representing physicalcharacteristics of signals of the moving image contents, extracts, fromthe structural search information obtained by the division, informationcorresponding to the semantic search information and the signal searchinformation, respectively, and forms small units of search informationby adding the extracted structural information to the semantic searchinformation and the signal search information obtained by the division,respectively.

The second search information dividing unit divides the searchinformation along time section based on the tree structure, and forms,from divided search information, small units of search information.

The division selecting unit selects whether the search information is tobe divided by the first search information dividing unit or the secondsearch information dividing unit, in accordance with the desired mannerof search.

Therefore, when the search information having the tree structurecorresponding to the moving image contents is to be divided into smallunits of search information and managed, it is possible to selectivelyform, in accordance with the desired manner of search, a small unit ofsearch information in accordance with one format or a small unit ofsearch in accordance with the other format. The one format is associatedwith the type of the search information. Specifically, according to thisformat, the search information is divided into structural searchinformation, semantic search information and signal search information,from the structural search information obtained by the division,information corresponding to the semantic search information and thesignal search information are extracted respectively, and the extractedstructural information is added to the semantic search information andthe signal search information respectively. In the other format, thesearch information is divided along time section, and the unitinformation consists of the thus divided search information. As it ispossible to select whether the small unit of search information is to beformed in accordance with the one format or the other format, theoptimal small unit of search information can be selected dependent onthe manner of search, such as an on-line search or an off-line search.Thus, a highly efficient search is realized.

According to a still further aspect of the present invention, the searchinformation managing apparatus includes, for managing the searchinformation having the tree structure corresponding to moving imagecontents divided into a small units of search information, first andsecond search information forming units and a selecting unit.

The first search information forming unit separates and draws out, fromthe search information, one or a plurality of search informationelements given by one or a plurality of leaves of the tree structure orgiven by one or a plurality of sub-trees consisting of an arbitrary setof a node and leaves or nodes of lower levels linked to the node of thetree structure, representing elements of the same type included in thesearch information, extracts, from the search information, structuralinformation corresponding to the drawn out one or a plurality of searchinformation elements, and forms a small unit of search information byarranging the one or a plurality of search information elements and theextracted corresponding structural information in a prescribed format.

The second search information forming unit divides the searchinformation along time section based on the tree structure, and forms,from the divided search information, small units of search information.

The selecting unit selects whether the small unit of search informationis to be formed by the first search information forming unit or thesecond search information forming unit, in accordance with the desiredmanner of search.

Therefore, when the search information having the tree structurecorresponding to the moving image contents is to be divided into smallunits of search information and managed, it is possible to form, inaccordance with the desired manner of search, a small unit in accordancewith one format or the other format. The one format is associated withthe type of the search information. Specifically, according to thisformat, one or a plurality of search information elements given by oneor a plurality of leaves of the tree structure or given by one or aplurality of sub-trees consisting of an arbitrary set of a node andleaves or nodes of lower levels linked to the node of the treestructure, representing elements of the same type included in the searchinformation are drawn out, structural information corresponding to thedrawn out one or a plurality of search information elements isextracted, and the one or a plurality of search information elements andthe corresponding structural information are arranged in a prescribedformat. In the other format, the search information is divided alongtime section, and the unit information is formed by the thus dividedsearch information. As it is possible to select whether the small unitof search information is to be formed in accordance with the one formator the other format, the optimal small unit of search information can beselected dependent on the manner of search, such as an on-line search oran off-line search. Thus, a highly efficient search is realized.

According to a still further aspect of the present invention, a searchinformation managing method includes, for managing search informationhaving a tree structure corresponding to moving image contents dividedinto small units of search information, the separating step, thestructural information extracting step and the unit search informationforming step.

In the separating step, the search information is separated intostructural search information representing the structure of the movingimage contents, semantic search information representing characteristicsin semantic context of the moving image contents, and signal searchinformation representing physical characteristics of signals of themoving image contents and output.

In the structural information extracting step, the search information isdisassembled, and from the disassembled structural search information,information corresponding to at least one of semantic search informationand signal search information is extracted as the structuralinformation.

In the unit search information forming step, a small unit of searchinformation is formed by adding, to at least one of the semantic searchinformation and signal search information separated from the searchinformation, the corresponding structural information extracted in thestructural information extracting step.

Therefore, a small unit of search information, provided by adding to atleast one of the semantic search information and signal searchinformation output from the separating unit the corresponding structuralinformation extracted from the structural information extracting unit,can be used as the search information for the moving image contents, andhence, the amount of search information can be reduced. Accordingly,when a small unit of search information requested by a searcher at thetime of searching for moving image contents is to be formed andtransmitted to the source of request, the amount of search informationto be transmitted can be reduced.

According to a still further aspect of the present invention, a searchinformation managing method includes, for managing search informationhaving a tree structure corresponding to moving image contents dividedinto small units of search information, the separating step, thestructural information extracting step and the unit search informationforming step.

In the separating step, one or a plurality of search informationelements given by one or a plurality of leaves of the tree structure orgiven by one or a plurality of sub-trees consisting of an arbitrary setof a node and leaves or nodes of lower levels linked to the node of thetree structure, representing elements of the same type included in thesearch information, are separated and drawn out from the searchinformation. In the structural information extracting step, structuralinformation corresponding to the one or a plurality of searchinformation elements drawn out by the separating step is extracted fromthe search information. In the unit search information forming step, asmall unit of search information is formed by arranging, in a prescribedformat, one or a plurality of search information elements drawn out bythe separating step and the corresponding structural informationextracted by the structural information extracting step.

Thus, a small unit of search information is formed by arranging the oneor a plurality of search information elements drawn out from theoriginal search information and the corresponding structural informationin accordance with a prescribed format, and therefore, each of thesearch information elements included in the small unit of searchinformation can easily be specified. Accordingly, the small unit ofsearch information can easily be used at the time of executing a searchfor moving image contents.

In accordance with a still further aspect of the present invention, asearch information managing method includes, for forming and managingsmall units of search information from search information having a treestructure corresponding to moving image contents, the record positioninformation extracting step, the structural information extracting stepand the unit search information forming step.

In the record position information extracting step, for each of one or aplurality of search information elements given by one or a plurality ofleaves of the tree structure or given by one or a plurality of sub-treesconsisting of an arbitrary set of a node and leaves or nodes of lowerlevels linked to the node of the tree structure, representing elementsof the same type included in the search information, record positioninformation representing the position where the search informationelement is recorded is extracted. In the structural informationextracting step, structural information corresponding to the one or aplurality of search information elements is extracted from the searchinformation. In the unit search information forming step, a small unitof search information is formed by arranging, in a prescribed format,the record position information corresponding to the one or a pluralityof search information elements extracted by the record positioninformation extracting step and the structural information extracted bythe structural information extracting step.

Therefore, the one or a plurality of search information elementsrepresented by the small unit of search information can be specified bythe record position information arranged in the small unit of searchinformation, and the search information elements can easily be used atthe time of executing a search for the moving image contents. When asmall unit of search information is requested for searching for movingimage contents and the small unit of search information is transmittedto the source of request, it is possible for the source of request tospecify the desired search information element by the record positioninformation in the received small unit of search information without thenecessity of transmitting the search information element itself.Further, by transmitting only the desired search information element atevery request to the source of request, a highly efficient searchbecomes possible.

According to a still further aspect of the present invention, the searchinformation managing method includes, for managing search informationhaving a tree structure corresponding to moving image contents dividedinto small units of search information, first and second searchinformation dividing step and the division selecting step.

In the first search information dividing step, the search information isdivided into structural search information representing the structure ofthe moving image contents, semantic search information representingcharacteristics in semantic context of the moving image contents, andsignal search information representing physical characteristics ofsignals of the moving image contents, from the divided structural searchinformation, information corresponding to the semantic searchinformation or signal search information, respectively, is extracted,and small units of search information are formed by adding the extractedstructural information to the semantic search information and the signalsearch information obtained by the division, respectively.

In the second search information dividing step, the search informationis divided along time section based on the tree structure, and fromdivided search information, small units of search information areformed.

In the division selecting step, whether the search information is to bedivided by the first search information dividing step or the secondsearch information dividing step is selected, in accordance with thedesired manner of search.

Therefore, when the search information having the tree structurecorresponding to the moving image contents is to be divided into smallunits of search information and managed, it is possible to selectivelyform, in accordance with the desired manner of search, a small unit ofsearch information in accordance with one format or a small unit ofsearch in accordance with the other format. The one format is associatedwith the type of the search information. Specifically, according to thisformat, the search information is divided into structural searchinformation, semantic search information and signal search information,from the structural search information obtained by the division,information corresponding to the semantic search information and thesignal search information are extracted respectively, and the extractedstructural information is added to the semantic search information andthe signal search information respectively. In the other format, thesearch information is divided along time section, and the unitinformation consists of the thus divided search information. As it ispossible to select whether the small unit of search information is to beformed in accordance with the one format or the other format, theoptimal small unit of search information can be selected dependent onthe manner of search, such as an on-line search or an off-line search.Thus, a highly efficient search is realized.

In accordance with a still further aspect of the present invention, thesearch information managing method is for managing search informationhaving a tree structure corresponding to moving image contents dividedinto small units of search information, and it includes the firstforming step, the second forming step and the selecting step. In thefirst forming step, one or a plurality of search information elementsgiven by one or a plurality of sub-trees are separated and drawn outfrom the search information, structural information corresponding to thedrawn out one or a plurality of search information elements is extractedfrom the search information, and by arranging the drawn out one or aplurality of search information elements and the extracted correspondingstructural information in a prescribed format, a small units of searchinformation is formed. The aforementioned sub-tree consists of leaves inthe tree structure or an arbitrary set of a node of the tree structureand leaves or nodes of the lower levels linked to the nodes,representing the elements of the same type included in the searchinformation.

In the second forming step, the search information is divided along timesection based on the tree structure, and from each of the divided searchinformation, a small unit of search information is formed.

In the selecting step, whether the small unit of search information isto be formed by the first forming step or the second forming step isselected, in accordance with the desired manner of search.

Therefore, when the search information having the tree structurecorresponding to the moving image contents is to be divided into smallunits of search information and managed, it is possible to form, inaccordance with the desired manner of search, a small unit in accordancewith one format the other format. The one format is associated with thetype of the search information. Specifically, according to this format,one or a plurality of search information elements given by the abovedescribed one or a plurality of sub-trees are drawn out, structuralinformation corresponding to the drawn out one or a plurality of searchinformation elements is extracted, and the one or a plurality of searchinformation elements and the corresponding structural information arearranged in a prescribed format. In the other format, the searchinformation is divided along time section, and the unit information isformed by the thus divided search information. As it is possible toselect whether the small unit of search information is to be formed inaccordance with the one format or the other format, the optimal smallunit of search information can be selected dependent on the manner ofsearch, such as an on-line search or an off-line search. Thus, a highlyefficient search is realized.

In accordance with a still further aspect of the present invention, thesearch apparatus includes, for conducting a search based on a small unitof search information obtained by dividing search information having atree structure corresponding to moving image contents, a holding unitholding small units of search information and a search executing unitexecuting a search for the moving image contents using the small unitsof search information. The small unit of search information is at leastone of small units of search information formed of semantic searchinformation representing characteristics in semantic context of themoving image contents, or signal search information representingphysical characteristics of signals of the moving image contents, andincludes structural information obtained from the structural searchinformation representing the structure of the moving image contents.

The small unit of search information is formed including at least one ofsemantic search information representing characteristics in semanticcontext of the moving image contents and signal search informationrepresenting physical characteristics of signals of the moving imagecontents, and the structural information of the moving image contents.Therefore, it is possible to conduct a search using a small unit ofsearch information corresponding to desired information of semanticsearch information or signal search information. Thus, an efficientsearch becomes possible.

In accordance with a still further aspect of the present invention, thesearch apparatus includes, for conducting a search based on a small unitof search information obtained by dividing search information having atree structure corresponding to moving image contents, a holding unitholding small units of search information and a search executing unitexecuting a search for the moving image contents using the small unitsof search information. The small unit of search information is formed byarranging, in a prescribed format, one or a plurality of searchinformation elements given by one or a plurality of leaves of the treestructure or given by one or a plurality of sub-trees consisting of anarbitrary set of a node and leaves or nodes of lower levels linked tothe node of the tree structure, representing elements of the same typeincluded in the search information and separated and drawn out from thesearch information as well as structural information corresponding tothe one or a plurality of search information elements extracted from thesearch information.

Therefore, a search is conducted using a small unit of searchinformation formed by drawing out one or a plurality of searchinformation elements given by one or a plurality of leaves of the treestructure or given by one or a plurality of sub-trees consisting of anarbitrary set of a node and leaves or nodes of lower levels linked tothe node of the tree structure, representing elements of the same typeincluded in the search information, extracting structural informationcorresponding to the drawn out one or a plurality of search informationelements and arranging the one or a plurality of search informationelements and the corresponding structural information in a prescribedformat. Accordingly, each of the search information elements included inthe small unit of search information can easily be specified, and ahighly efficient search is possible.

In accordance with a still further aspect of the present invention, thesearch apparatus for executing a search based on small units of searchinformation formed from search information having a tree structurecorresponding to moving image contents includes a first holding unitholding small units of search information, a second holding unit holdingone or a plurality of search information elements obtained by using thesmall units of search information, and a search executing unit executinga search for the moving image contents, using the small units of searchinformation and the one or a plurality of search information elements.The small unit of search information is formed, for each of the one or aplurality of search information elements given by one or a plurality ofleaves of the tree structure or given by one or a plurality of sub-treesconsisting of an arbitrary set of a node and leaves or nodes of lowerlevels linked to the node of the tree structure, representing elementsof the same type included in the search information, by arranging, in aprescribed format, record position information representing the positionwhere the corresponding search information element is recorded and thecorresponding structural information extracted from the searchinformation.

Therefore, a search is conducted using small units of search informationformed by arranging, in a prescribed format, record position informationand the structural information for each of the one or a plurality ofsearch information elements given by one or a plurality of leaves of thetree structure or given by one or a plurality of sub-trees consisting ofan arbitrary set of a node and leaves or nodes of lower levels linked tothe node of the tree structure, representing elements of the same typeincluded in the search information. Accordingly, the search informationelement designated by the record position information in the small unitof search information can easily be specified. Further, by obtainingonly the minimum necessary search information element as needed inaccordance with the situation, a highly efficient search is possible.

In accordance with a still further aspect of the present invention, thesearch information managing apparatus includes a position calculatingunit and a recovering unit.

In order to recover, from a small unit of search information obtained bydividing search information having a tree structure corresponding tomoving image contents, part of the search information having the treestructure, the position calculating unit calculates a position in thetree structure that corresponds to semantic search information or signalsearch information included in the small unit of search information fromthe structural information included in the small unit of searchinformation.

The recovering unit recovers, from the semantic search information orthe signal search information included in the small unit of searchinformation and the calculated position of the tree structure, part ofthe search information having the tree structure.

The small unit of search information is at least one of small units ofsearch information formed of semantic search information representingcharacteristics in semantic context of the moving image contents andsignal search information representing physical characteristics ofsignals of the moving image contents, and includes structuralinformation obtained from structural search information representing thestructure of the moving image contents.

Therefore, the small unit of search information is formed including atleast one of semantic search information representing characteristics insemantic context of the moving image contents and signal searchinformation representing physical characteristics of signals of themoving image contents, and structural information of the moving imagecontents. Therefore, by recovering a part of the search informationhaving the tree structure utilizing the small unit of search informationcorresponding to the desired semantic search information or signalsearch information, it becomes possible to manage only a small amount ofsearch information in the form of a tree.

In accordance with a still further aspect of the present invention, thesearch information managing apparatus includes a position calculatingunit and a recovering unit.

In order to recover, from a small unit of search information obtained bydividing search information having a tree structure corresponding tomoving image contents, part of the search information having the treestructure, the position calculating unit calculates a position in thetree structure that corresponds to one or a plurality of searchinformation elements included in the small unit of search information,from the structural information included in the small unit of searchinformation.

The recovering unit recovers, from the one or a plurality of searchinformation elements included in the small unit of search informationand the calculated position of the tree structure, part of the searchinformation having the tree structure.

The small unit of search information is formed by arranging, in aprescribed format, one or a plurality of search information elementsgiven by one or a plurality of sub-trees and structural informationcorresponding to the one or a plurality of search information elementsextracted from the search information. Here, the sub-tree consists ofleaves of the structure or an arbitrary set of a node and leaves ornodes of lower levels linked to the node of the tree structure,representing elements of the same type included in the searchinformation.

Therefore, the small unit of search information is formed by arranging,in a prescribed format, one or a plurality of search informationelements given by one or a plurality of sub-trees and the correspondingstructural information.

Therefore, by recovering a part of the search information having thetree structure utilizing the small unit of search informationcorresponding to the desired search information element, it becomespossible to manage only a small amount of information in the form of atree structure.

The search information managing apparatus in accordance with a stillfurther aspect of the present invention includes a position calculatingunit, a search information element obtaining unit and a recovering unit.

Position calculating unit calculates, based on the small unit of searchinformation formed from the search information having the tree structurecorresponding to the moving image contents, a position or positions ofone or a plurality of search information elements indicated by therecord position information included in the small unit of searchinformation from the structural information included in the small unitof search information, in order to recover part of the searchinformation having the tree structure.

Search information obtaining unit obtains, based on the record positioninformation included in the small unit of search information, thecorresponding one or a plurality of search information elements. Therecovering unit recovers part of the search information having the treestructure, from the obtained one or a plurality of search informationelements and the obtained position in the tree structure.

The small unit of search information is formed by arranging, for each ofthe one or a plurality of search information elements given by one or aplurality of sub-trees, record position information indicating theposition where the search information element is recorded, and thecorresponding structural information extracted from the searchinformation. Here, the sub-tree consists of leaves of the structure oran arbitrary set of a node and leaves or nodes of lower levels linked tothe node of the tree structure, representing elements of the same typeincluded in the search information.

Therefore, the small unit of search information is formed by arranging,in a prescribed format, record position information of respective onesof the one or a plurality of search information elements given by theabove described one or a plurality of sub-trees included in the searchinformation and the corresponding structural information. Therefore, byrecovering a part of the search information having the tree structureutilizing the small unit of search information including the recordposition information corresponding to the desired search informationelement, it becomes possible to manage only a small amount ofinformation in the form of a tree structure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a search information managing apparatusin accordance with the first and second embodiments of the presentinvention.

FIG. 2 represents search information having a tree structure used in thefirst embodiment of the present invention.

FIG. 3 shows an internal configuration of the semantic unit searchinformation forming unit and the signal unit search information formingunit in the search information managing apparatus in accordance with thefirst embodiment of the present invention.

FIGS. 4A to 4C represent unit search information of the semantic searchinformation obtained by division in accordance with the first embodimentof the present invention.

FIGS. 5A to 5C represent unit search information of the signal searchinformation obtained by division in accordance with the first embodimentof the present invention.

FIG. 6 is a flow chart representing the search information managingmethod in accordance with the first embodiment of the present invention.

FIG. 7 is a schematic diagram of the search apparatus in accordance withthe first and second embodiments of the present invention.

FIGS. 8A to 8D represent unit search information of semantic searchinformation and signal search information, respectively, formed inaccordance with the second embodiment of the present invention.

FIG. 9 is a schematic diagram of the search information managingapparatus in accordance with the third embodiment of the presentinvention.

FIG. 10 illustrates a general structure of moving image contents.

FIG. 11 represents, in a tree structure, search information added to themoving image contents of FIG. 10.

FIG. 12 illustrates the method of recording the search information ofthe moving image contents shown in FIG. 10.

FIGS. 13A to 13D represent conventional division of the searchinformation having the tree structure.

FIG. 14 represents a conventional method of managing search informationhaving the tree structure.

BEST MODES FOR CARRYING OUT THE INVENTION

Embodiments of the present invention will be described with reference tothe drawings.

First Embodiment

FIG. 1 shows a functional configuration of a search information managingapparatus 1 in accordance with the present embodiment. Referring to FIG.1, search information managing apparatus 1 includes a search informationanalyzing unit 101, a structural search information disassembling unit102, a semantic unit search information forming unit 103, a signal unitsearch information forming unit 104 and a unit search informationcontrol unit 105. Search information analyzing unit 101 receives, as aninput, search information 15 having a tree structure corresponding tomoving image contents, separates the input search information 15 intoone or more search information elements 152 of the structural searchinformation, one or more search information elements 153 of the semanticsearch information and one or more search information elements 154 ofthe signal search information, and selectively outputs the same.

Structural search information disassembling unit 102 receives one ormore structural search information elements 152 output from searchinformation analyzing unit 101, draws out minimum necessary structuralinformation 50 from the received search information elements 152, andoutputs the same to semantic unit search information forming unit 103and signal unit search information forming unit 104. More specifically,structural search information disassembling unit 102 receives as inputsthe applied one or more structural search information elements,disassembles the same, and from the contents of disassembled one or morestructural search information elements, extracts, as minimum structuralinformation 50, information corresponding to at least one of thesemantic search information and the signal search information.

FIG. 2 represents the search information having the tree structure usedin the present embodiment. For simplicity of description, it is assumedthat in the present search information 15, structural search informationelement 152 consists only of node ID information and time information ofeach scene and each shot, semantic search information element 153consists only of visual scenic description information of each scene andcharacter information and action description information of each shot,and signal search information element 154 consists only ofrepresentative color information of each scene and color frequencyinformation and motion intensity information of each shot. Actual searchinformation 15 additionally has various search information elements, andthe present invention can similarly be applied to the various searchinformation elements.

Structural search information disassembling unit 102 selects and drawsout, as the minimum necessary structural information 50, either one ofor both of structuring information (such as node ID) 50A and structuralattribute information (such as time information) 50B. Which informationis to be selected as structural information 50 depends on the manner ofsearch that will be executed using of the resulting unit searchinformation.

When structuring information 50A is selected, a search can be executedfocusing on the hierarchical structure of the moving image contents orrelation between each of the scenes or shots of the entire moving imagecontents. When the structural attribute information 50B is selected, asearch can be executed in which a portion in the moving image contentsthat corresponds to the result of search can immediately be drawn outand displayed.

The structuring information 50A will be described in grater detail.Structuring information 50A represents information for forming the treestructure of the search information, that enables recovery of theoriginal tree structure. The node ID information as the structuringinformation 50A of the present embodiment represents the linkagerelation between the nodes (for example, “#1-1” and “#1-2” representchildren nodes ND that are linked to “#1”). Using this information, theoriginal tree structure can be recovered. Therefore, in a searchrequiring recovery of the original tree structure, the structuringinformation 50A is selected as the structural information.

Though ID information is not added to a leaf RF that represents eachsearch information element in FIG. 2, the ID information may be added toeach leaf RF. Other structuring information 50A may include a codeallocated to each “branch” (the bar portion connecting the root R, nodeND and leaf RF with each other) of the tree structure, so that by thecodes of branches starting from the root R to a certain node ND or leafRF, the position information of the node ND or the leaf RF can beobtained.

Structural information 50 extracted from structural search informationelement 152 may have redundant information omitted and the informationmay be compressed as needed, in order to provide minimum necessaryinformation. By way of example, when node ID information as thestructuring information 50A is used as the structural information 50,“#1-2” following “#1-1” has the node ID “#1” of the upper node ND commonto the preceding one, and therefore, only the information “2” may beapplied as the structural information 50 with the common “#1” omitted.When it is known that “1” is always followed by “2” (always incrementedby 1), difference information representing “increment by 1” may begenerated and used as structural information 50. The method ofextracting the minimum necessary information is not limited to theabove.

Semantic unit search information forming unit 103 receives as inputssemantic search information element 153 applied from search informationanalyzing unit 101 and the minimum necessary structural information 50applied from structural search information disassembling unit 102,arranges the input information and forms unit search information 153A ofsemantic search information.

FIG. 3 shows an internal configuration of semantic unit informationforming unit 103 of FIG. 1. Referring to FIG. 3, semantic unit searchinformation forming unit 103 includes an element type determining unit201, a search information arranging unit 202 and a shared informationextracting unit 203. Here, search information elements 153 (leaves RF)of one same type connected to nodes ND (or root R) of the same type inthe tree structure of search information 15 are arranged, type by typeof the search information elements 153, while maintaining relation ofappearance order thereof in the tree structure, to form unit searchinformation 153A. Specifically, element type determining unit 201receives search information elements 153 of semantic search informationapplied from search information analyzing unit 101, and whiledetermining the types (representing different types such as characterinformation, action description information and the like) of thereceived search information elements, outputs each of the receivedsearch information elements 153 together with the result ofdetermination, to search information arranging unit 202 and to sharedinformation extracting unit 203. Search information arranging unit 202arranges search information elements 153 type by type based on theinformation applied from element type determining unit 201, togetherwith the corresponding minimum necessary structural information 50 drawnout by structural search information disassembling unit 102, to formunit search information 153A. At this time, information 153B (forexample, information representing the type of search information element153) shared by search information elements 153 included in respectiveones of unit of search information 153A formed type by type by searchinformation arranging unit 202 is extracted by shared informationextracting unit 203 and applied to search information arranging unit202. Search information arranging unit 202 places shared information153B applied from shared information extracting unit 203 at the head ofthe formed unit search information 153A. Other than this example,information representing format or data length of semantic searchinformation elements 153 may be used as the shared information 153B, andinformation used as the shared information 153B is not specificallylimited.

From the structural information 50 drawn out from structural searchinformation disassembling unit 102 or from the original structuralsearch information element 152, information that can be shared, such asthe level information where search information elements are connected tonodes of the same level, may be extracted by shared informationextracting unit 203 and may be placed at the head of the unit searchinformation, as the. appended shared information 153B.

Further, based on the level information of nodes ND obtained fromstructural information 50 or structural search information element 152,it is possible to distinguish semantic search information elements 153of the same type but added to nodes ND of different levels from eachother, and to separately form unit search information 153A consistingonly of semantic search information elements 153 added to the samelevel.

As to the position of placing the shared information 153B, it is notlimited to the head of unit search information 153A, and it may beplaced at the tail of unit search information 153A or within unit searchinformation 153A. Alternatively, shared information 153B may be recordedat a different record position together with information representingcorrespondence between unit search information 153A and the sharedinformation 153B. Shared information 153B and the process by sharedinformation extracting unit 203 may be omitted.

FIGS. 4A to 4C represent unit search information 153A formed by drawingout visual scenic information, character information and actiondescription information as semantic search information elements 153 fromsearch information 15 having the tree structure shown in FIG. 2, and byarranging the drawn out semantic search information elements 153 andcorresponding structural information 50 together with shared information153B.

In unit search information 153A shown in FIGS. 4A to 4C, semantic searchinformation elements 153 and corresponding structural information 50 arepaired and arranged alternately as an example. The format of arrangementof semantic search information elements 153 and the correspondingstructural information 50 is not limited thereto. For example,structural information 50 may be arranged collectively at first, andcorresponding semantic search information elements 153 may becontinuously arranged thereafter, or the order may be reversed. There isno limit in the format of arrangement, except that the relation ofappearance order in the tree structure is maintained.

When shared information 153B of semantic search information elements 153is to be extracted from structural information 50 drawn out fromstructural search information disassembling unit 102 or from thestructural search information 152, difference information representing adifference from shared information 153B (information that cannot beshared) may be used as structural information 50 corresponding to eachof semantic search information elements 153.

When node ID information of structuring information 50A is used asstructural information 50, the unit search information 153A may beformed by the following procedure. By way of example, for a node NDhaving the node ID “#1-1” and a node ND having the node ID “#1-2”,information “#1” that corresponds to the node ID of a shared upper nodeND may be drawn out as shared information 153B and that one informationmay be added to unit search information 153A, while differenceinformation “1” and “2” of node ID, with the shared information “#1”omitted, may be arranged as structural information 50 corresponding torespective semantic search information elements 153.

Alternatively, taking one level as a reference, level information of thenodes ND thereof may be added as shared information, and informationrepresenting a difference from the reference level (informationrepresenting a level higher by one or lower by one) may be added to eachsemantic search information element 153. At that time, addition ofdifference information may be omitted for semantic search informationelement 153 that has no difference. Other than these, a flag indicatingwhether there is a semantic search information element 153 or notcorresponding to a certain scene or shot, or information representingimportance when semantic search information element 153 is used, may beused as the information to be added to each semantic search informationelement 153. The information to be added to each semantic searchinformation element 153 is not specifically limited.

Signal unit search information forming unit 104 receives one or moresearch information elements 154 of the signal search information fromsearch information analyzing unit 101, receives the minimum necessarystructural information 50 applied form structural search informationdisassembling unit 102, arranges the received search informationelements 154 and structural information 50 to form one or more units ofsearch information 154A of the signal search information, and outputsthe same. The internal configuration of signal unit search informationforming unit 104 is similar to that shown in FIG. 3. Here, signal searchinformation elements 154 (leaves RF) of one same type connected to nodesND (or root R) of the same type in the tree structure of searchinformation 15 are arranged, type by type of the signal searchinformation elements 154, while maintaining relation of appearance orderthereof in the tree structure, to form one or more units of searchinformation 154A. Specifically, element type determining unit 201 shownin FIG. 3 receives signal search information elements 154, determinesthe types (representing different types of information such as colorfrequency information and motion intensity information) of respectiveones of the received signal search information elements 154, and outputsthe result of determination and signal search information elements 154to search information arranging unit 202 and to shared informationextracting unit 203. Based on the applied information, searchinformation arranging unit 202 arranges the signal search informationelements 154 type by type, together with the corresponding minimumnecessary structural information 50 applied from structural searchinformation disassembling unit 102, to form and output one or more unitsof search information 154A. At this time, based on the appliedinformation, shared information extracting unit 203 extracts information154B (for example, information representing the type of signal searchinformation element 154) shared by signal search information elements154 included in respective ones of unit search information 154A formedby search information arranging unit 202 from signal search informationelements 154, and applies the same to search information arranging unit202. Search information arranging unit 202 places the shared information154B applied from shared information extracting unit 203 at the head ofunit search information 154A. Other than this example, informationrepresenting data format, sample number of color frequency informationor data length of signal search information elements 154 may be used asthe shared information 154B, and information used as the sharedinformation 154B is not specifically limited.

From the structural information 50 drawn out from structural searchinformation disassembling unit 102 or from the original structuralsearch information element 152, information that can be shared, such asthe level information where search information elements are connected tonodes of the same level, may be extracted by shared informationextracting unit 203 and may be placed at the head of the signal unitsearch information 154A, as the appended shared information 154B.

Further, based on the level information of nodes ND obtained from thestructural information 50 or structural search information 152, it ispossible to distinguish signal search information elements 154 of thesame type but added to nodes ND of different levels from each other, andto separately form unit search information 154A consisting only ofsignal search information elements 154 added to the same level.

As to the position of placing the shared information 154B, it is notlimited, and it may be placed at the tail of unit search information154A or within unit search information 154A. Alternatively, sharedinformation 154B may be recorded at a different record position togetherwith information representing correspondence between unit searchinformation 154A and the shared information 154B. Shared information154B and the process by shared information extracting unit 203 may beomitted.

Unit search information 154A shown in FIGS. 5A to 5C is formed bydrawing out, from search information 15 having the tree structure ofFIG. 2, the representative color information, color frequencyinformation and motion intensity information as the signal searchinformation elements 154, and arranging the drawn out signal searchinformation elements and the corresponding structural information 50together with the shared information 154B.

In unit search information 154A shown in FIGS. 5A to 5C, signal searchinformation elements 154 and corresponding structural information 50 arepaired and arranged alternately as an example. The format of arrangementof signal search information elements 154 and the correspondingstructural information 50 is not limited thereto. For example,structural information 50 may be arranged collectively at first, andcorresponding signal search information elements 154 may be continuouslyarranged thereafter, or the order may be reversed. There is no limit inthe format of arrangement, except that the relation of appearance orderin the tree structure is maintained.

When shared information 154B corresponding to signal search informationelements 154 is to be extracted from structural information 50 drawn outby structural search information disassembling unit 102 or from thestructural search information 152, difference information representing adifference from shared information 154B (information that cannot beshared) may be used as structural information 50 corresponding to eachof signal search information elements 154. When node ID information ofstructuring information 50A is used as structural information 50, theunit search information 154A may be formed by the following procedure.By way of example, for a node having the node ID “#1-1” and a node NDhaving the node ID “#1-2”, information “#1” that corresponds to the nodeID of a shared upper node ND may be drawn out as shared information 154Band that one information may be added to unit search information 154A,while difference information “1” and “2” of node ID, with the sharedinformation “#1” omitted, may be arranged as structural information 50corresponding to respective signal search information elements 154.

Alternatively, taking one level as a reference, level information of thenodes ND thereof may be added to unit search information 154A as sharedinformation 154B, and information representing a difference from thereference level (information representing a level higher by one or lowerby one) may be added to each signal search information element 154. Atthat time, addition of difference information may be omitted for signalsearch information element 154 that has no difference.

Other than these, a flag indicating whether there is a signal searchinformation element 154 or not corresponding to a certain scene or shot,or information representing importance when signal search informationelement 154 is used, may be used as the information to be added to eachsignal search information element 154. The information to be added toeach signal search information element 154 is not specifically limited.

Units of search information 153A and 154A of the semantic searchinformation and signal search information including structuralinformation 50 generated by semantic unit search information formingunit 103 and signal unit search information forming unit 104 are appliedto unit search information control unit 105.

Unit search information control unit 105 is connected to a processingunit 16 outside the search information managing apparatus 1. Processingunit 16 corresponds to a transmitting unit for transmitting information,a storing unit recording and storing information on some medium, or aportion for executing an information search process. When processingunit 16 is a transmitting unit, search information managing apparatus 1selects one or more units of search information meeting the request fromthe transmitting unit from units of search information 153A and 154A,and applies the same to processing unit 16 (transmitting unit).

Referring to the flow chart of FIG. 6, the search information managingmethod in accordance with the present embodiment will be described.First, search information 15 to be divided is designated, and searchinformation analyzing unit 101 receives as an input the designatedsearch information 15 (step S1: in the following, a step will be simplydenoted by S), separates the input search information 15 into searchinformation elements, and successively outputs structural searchinformation element 152, semantic search information element 153 andsignal search information element 154 obtained by the separation (S2).Whether the successively obtained search information elements correspondto the structural search information, semantic search information orsignal search information is determined, and based on the result ofdetermination, the search information elements are applied to any ofstructural search information disassembling unit 102, semantic unitsearch information forming unit 103 and signal unit search informationforming unit 104 (S3).

When the search information element represents structural searchinformation, structural information 50 that is necessary to form unitsof search information 153A and 154A of the semantic search informationand signal search information is extracted from the structural searchinformation element 152 (S4), and separation and reading of the searchinformation elements are repeated (S2).

When the search information element represents semantic searchinformation, the type thereof is determined (S5), and the semanticsearch information element 153 is placed at the tail of the arrangementof semantic search information elements 153 of the same type that havebeen arranged by that time, together with the already extractedstructural information 50 (S6).

When the search information element represents signal searchinformation, the type thereof is determined (S7), and the signal searchinformation element 154 is placed at the tail of the arrangement ofsignal search information elements 154 of the same type that have beenarranged by that time, together with the already extracted structuralinformation 50 (S8).

At the end of each of the processes for forming unit search information153A and 154A of steps S4 to S8, whether it is an end of searchinformation 15 or not is determined (S9). When it is determined to bethe end, the formed units of search information 153A and 154A arerespectively applied to unit search information control unit 105, andheld therein (S 10). If it is not the end, separation and reading ofsearch information elements are again repeated (S2).

By performing the process in accordance with the above describedconfiguration and flow chart, it becomes possible to transfer or useminimum information necessary to conduct a search for moving imagecontents, and a highly efficient search can be executed.

FIG. 7 represents a schematic configuration of a search apparatus 2 inaccordance with the present embodiment. As shown in FIG. 7, the searchapparatus 2 operated by a searcher 18 includes a search executing unit701 for executing a search for moving image contents, a semantic unitsearch information holding unit 702, a signal unit search informationholding unit 703 and a search information control unit 704. Searchexecuting unit 701 selects, in accordance with a search request input bythe searcher 18, an arbitrary combination of one or more of semanticunit search information 153A held (recorded) in semantic unit searchinformation holding unit 702 and signal unit search information 154Aheld (recorded) in signal unit search information holding unit 703,obtains unit search information of the selected combination, and usingthe obtained unit search information, executes a search for the movingimage contents. When the unit search information to be selected is notheld in any of corresponding unit search information holding units 702and 703, search executing unit 701 issues a request for thecorresponding unit search information to search information control unit704.

Search information control unit 704 is connected to a transmitting unitor to a storing unit 19. The transmitting unit or storing unit 19 has atransmitting portion for transmitting a plurality of types of unitsearch information, or a storing portion for recording and storing aplurality of types of unit search information on a prescribed medium inadvance. When a request for unit search information is input from searchexecuting unit 701, search information control unit 704 obtains, inresponse to the request, the unit search information corresponding tothe request from transmitting unit or storing unit 19 by transmission orreading, and stores the obtained unit search information tocorresponding one of the unit search information holding units 702 and703. Search executing unit 701 executes a search using the unit searchinformation obtained in this manner.

By way of example, when a method of search based on color characteristicof moving image contents is considered the most desirable, the searcher18 requests color frequency information. Alternatively, the searcher 18may simply input a broad request such as “information related to colordesired,” and in response to the request, search executing unit 701 maydetermine and appropriately “issue a request for color frequencyinformation.” To meet the request, unit search information 154Aconsisting of the color frequency information and the minimum necessarystructural information 50 is obtained from transmitting unit or storingunit 19.

Based on the unit search information 154A formed by the structuralinformation and the color frequency information obtained in this manner,search executing unit 701 executes a search. In this manner, the amountof search information transmitted at the time of a search can be reducedas compared with the conventional manner. The procedure of searchdescribed here is merely an example and not limiting.

The period how long the obtained unit search information is held is notspecifically limited. When the client terminal that executes a search isone having a small storage capacity such as a portable informationterminal, the obtained unit search information may be discarded everytime a search is conducted, and when the terminal has sufficient storagecapacity, the information may be continuously held while searches areconducted. Alternatively, such a management may be possible in that theunit search information may be held for a prescribed time period, andthe unit search information may be discarded if it is not accessed inthat period.

Though a search is executed based on the formed unit search informationin the search apparatus in accordance with the present embodiment, thesearch information of the original tree structure (part thereof) may berecovered from the obtained unit search information, and based on thesearch information of the recovered tree structure, a search may beexecuted, dependent on the manner or search.

As each of the units of search information formed by the searchinformation managing apparatus in accordance with the present inventionincludes the structural information, it is possible to recover theoriginal tree structure by using the structural information (structuringinformation). Specifically, by finding the position information in thetree structure from the structural information included in the unitsearch information, obtaining the corresponding search informationelement indicated by the corresponding record position information inthe unit search information, and connecting the same as a leaf RF to theposition of the tree structure indicated by the found positioninformation, the tree structure is recovered. The recovery from the unitsearch information to the tree structure can be used not only for thesearch but also for transmission of the search information to otherperson through a narrow network, by dividing the tree into small pieces,transmitting each of the divided pieces to the destination andrecovering the tree at the receiving side to be managed as the searchinformation.

Each unit search information (153A, 154A) shown in FIGS. 4A to 4C and 5Ato 5C is formed by arranging corresponding search information elements(153, 154) respectively in order, and each of the search informationelements may be coupled, or each of the search information elements maybe held in an independent format. In the latter case, it is possible tooutput only the necessary number of search information elements (153,154) as partial unit search information (153A, 154A).

In FIGS. 4A to 4C and 5A to 5C, all the search information elements(153, 154) of the same type included in search information 15 shown inFIG. 2 are used to form units of search information (153A, 154A). Theunits of search information, however, may be formed by using only thepart of search information elements. The part may include a subset(subset as compared with the whole) of search information elementsneighboring successively in time, or a set of sub samples picked-up at aprescribed interval.

Each of the search information 15 and search information elements (152,153, 154) described in the present embodiment is an example only, andthe search information and search information elements are not limitedthereto. The tree structure of search information 15 may have a largernumber of levels, or it may consist of two levels (only the root R andone or more nodes ND of the first level). The tree structure may haveany shape. The present embodiment is applicable regardless of the typesof search information elements.

In the present embodiment, though the search information elementscollected as the unit search information is described as individualinformation corresponding to each “leaf” of the tree structure such asthe color frequency information, it is not limiting. For example, “colorinformation” collectively including a plurality of pieces of informationrelated to “color” such as color frequency information andrepresentative color information may be considered as one element, andthe unit search information may be formed regarding the one element asthe search information element.

Here, each piece of color frequency information or representative colorinformation corresponds to “leaf RF” of the tree structure, and “colorinformation” corresponds to a “node ND” having the leaves RF at thelower level. Specifically, in the present embodiment, respective searchinformation elements forming the unit search information are not limitedto detailed pieces of search information corresponding to the “leavesRF” of the tree structure, and they may be an arbitrary set of detailedpieces of search information below a “node ND” that can be regarded as agroup.

Here, an arbitrary set of leaves RF or nodes ND linked at a lower levelto a certain node ND (with each of the nodes ND further linked to leavesRF or nodes ND of lower level) forms a tree with the starting nodeconsidered as a root R, and this tree is referred to as a “partialtree”, or “sub-tree”, of the original tree. In other words, the presentembodiment may be applied with the portion of the search informationthat corresponds to the “partial tree” of the tree structurerepresenting the search information considered as the search informationelement.

In the search information managing apparatus 1 and the searchinformation managing method in accordance with the present embodiment,all the search information elements (153, 154) included in the searchinformation 15 having the tree structure are drawn out in the similarmanner, all the units of search information (153A, 154A) correspondingto the drawn out search information elements respectively are formed inparallel, and all the thus formed units of search information aremanaged for transmission, for storage or for a search. It is alsopossible, however, to draw out only the search information elements of aspecific type and form only a unit of search information correspondingto the drawn out specific type of search information element.

By way of example, only the color frequency information may be drawn outamong the search information elements, and only the unit searchinformation corresponding to the drawn out color frequency informationmay be formed and managed by unit search information control unit 105.In that case, search information analyzing unit 101, structural searchinformation disassembling unit 102, signal unit search informationforming unit 104 and unit search information control unit 105 onlyfunction. Therefore, semantic unit search information forming unit 103may be omitted.

The same applies to search apparatus 2 in accordance with the presentembodiment. When a search is executed based particularly on the colorfrequency information among the signal search information, and thesemantic search information is deemed unnecessary, only the searchexecuting unit 701, signal unit search information holding unit 703 andsearch information control unit 704 of search apparatus 2 shown in FIG.7 function. Therefore, semantic unit search information holding unit 702may be omitted.

When part of the semantic search information elements 153 or signalsearch information elements 154 is drawn out and corresponding unitsearch information (153A or 154A) is formed using the drawn out part ofthe search information elements (153 or 154), the remaining searchinformation 15 from which the search information elements (153 or 154)are drawn out may be utilized while maintaining the tree structure.

Specifically, here, the tree representing the original searchinformation 15 is divided into a partial tree as a part of the originaltree and one or a plurality of units of search information 153Aconsisting of a plurality of semantic search information elements 153 orone or a plurality of units of search information 154A consisting of aplurality of signal search information elements 154. In the presentinvention, dependent on which search information element is selected tobe drawn out, original search information 15 having the tree structurecan be divided in various shapes.

In the present embodiment, structural information of the searchinformation, that is, position information of a node ND or a leaf RF inthe tree structure, and type information of the search informationelement, that is, information representing that the element is actiondescription information (semantic search information element 153) orcolor frequency information (signal search information element 154) aredescribed as independent pieces of information. When the searchinformation elements forming the search information are determined inadvance and the tree structure is fixed, the position of each searchinformation element in the tree structure can be specified, andtherefore, the type of the search information element can be representedsimply by the structural information. In other words, the structuralinformation and the type information are equivalent. Here, the processof adding type information as shared information 153B and 154Bcorresponding to search information elements 153 and 154 described withreference to the present embodiment, for example, can be realized byadding the corresponding shared structural information.

Second Embodiment

In the first embodiment, various search information elements, that is,structural search information element 152, semantic search informationelement 153 and signal search information element 154 are separated anddrawn out from search information 15 having a tree structure, structuralinformation 50 is extracted from the drawn out structural searchinformation element 152, and by arranging the semantic searchinformation elements 153 of the same type themselves or the signalsearch information elements 154 of the same type themselves togetherwith the extracted structural information 50 in order, semantic unitsearch information 153A or signal unit search information 154A isformed.

In contrast, in the present embodiment, where semantic searchinformation element 153 and signal search information element 154 areseparated in advance from the search information 15 and recordedindependently, not the semantic search information element 153 andsignal search information element 154 themselves but informationrepresenting record positions of these is obtained, and the obtainedrecord position information is arranged in order together with thestructural information, to form semantic unit search information 153Eand signal unit search information 154E.

By this configuration, similar effects as the first embodiment can beobtained. In the following, a search information managing apparatus 1Aand a search apparatus 2A in accordance with the present embodiment willbe described, focusing on the difference from the corresponding elementsof the first embodiment. Therefore, details of the present embodimentare not described here, and detailed descriptions related to the searchinformation managing apparatus 1 and method as well as to the searchapparatus 2 in accordance with the first embodiment are similarlyapplicable.

The schematic configuration of the search information managing apparatus1A in accordance with the present embodiment is shown, similar to thefirst embodiment, in FIG. 1. The function of search information managingapparatus 1A is slightly different from that of the first embodiment, aswill be described below.

Referring to FIG. 1, search information managing apparatus 1A includessearch information analyzing unit 101A, structural search informationdisassembling unit 102, semantic unit search information forming unit103A, signal unit search information forming unit 104A and unit searchinformation control unit 105A. Search information analyzing unit 101Aanalyzes search information 15 having the tree structure correspondingto moving image contents, outputs structural search information element152 included in search information 15 to structural search informationdisassembling unit 102, specifies semantic search information element153 and signal search information element 154, respectively, andextracts, for respective ones of the specified search informationelements 153 and 154, record position information 153D and 154D. Recordposition information 153D and 154D represent record positions wherecorresponding semantic search information element 153 and signal searchinformation element 154 are recorded, respectively. Thereafter, togetherwith information necessary to form units of search information 153E and154E, for example, information of the types of search informationelements and level information of the tree structure, the recordposition information 153D and 154D of search information elements 153and 154 respectively are output to corresponding semantic unit searchinformation forming unit 103A and signal unit search information formingunit 104A, respectively.

Record position information 153D and 154D may include but not limited tophysical position information such as a sector position of a recordingmedium or a bit position in a bit sequence on which the searchinformation 15 is recorded in advance, or logical information such as afile name containing the data, a channel for data transmission or apacket.

Semantic search information element 153 and signal search informationelement 154 indicated by record position information 153D and 154D areindependently read and utilized as needed at a time of a search formoving image contents, for example. Therefore, it may be convenient thatthe semantic search information element 153 and the signal searchinformation element 154 are separated in advance and recordedindependently. In many practical servers, structural search informationelement 152, semantic search information element 153 and signal searchinformation element 154 of search information 15 having the treestructure are separated and recorded in some way or other. Therefore,record position of such divided semantic search information element 153and signal search information element 154 can directly be utilized asrecord position information 153D and 154D.

When the search information 15 having the tree structure is recorded asit is and not separated in advance into search information elements 152,153 and 154, search information analyzing unit 101A separates searchinformation elements 152, 153 and 154, records search informationelements 153 and 154, and generates record position information 153D and154D thereof.

Structural search information disassembling unit 102 receives as aninput the structural search information element 152 from searchinformation analyzing unit 101A, draws out minimum necessary structuralinformation 50 from the input search information element 152, andapplies the same to semantic unit search information forming unit 103Aand signal unit search information forming unit 104A.

Semantic unit search information forming unit 103A receives as inputsthe minimum necessary structural information 50 from structural searchinformation disassembling unit 102 and record position information 153Dof each of semantic search information elements 153 of the same typefrom search information analyzing unit 101A, and arranges the inputinformation type by type of the semantic search information elements153, while maintaining the relation of appearance order in the treestructure of the corresponding semantic search information elements, toform the unit search information 153E of the semantic search informationconsisting of the record position information 153D of the semanticsearch information elements 153.

Signal unit search information forming unit 104A receives as inputs theminimum necessary structural information 50 from structural searchinformation disassembling unit 102 and record position information 154Dof each of signal search information elements 154 of the same type fromsearch information analyzing unit 101A, and arranges the inputinformation type by type of the signal search information elements 154,while maintaining the relation of appearance order in the tree structureof the corresponding signal search information elements, to form theunit search information 154E of the signal search information consistingof the record position information 154D of the signal search informationelements 154.

FIGS. 8A to 8D represent, as examples, semantic unit search information153E and signal unit search information 154E in accordance with thepresent invention. In FIGS. 8A and 8B, visual scenic descriptioninformation of respective scenes and character information of respectiveshots are shown as examples of semantic search information elements 153,and in FIGS. 8C and 8D, representative color information of respectivescenes and motion intensity information of respective shots are shown asexamples of signal search information elements 154.

As is apparent from the comparison between FIGS. 8A to 8D, 4A to 4C and5A to 5C, the unit search information (153E and 154E) formed inaccordance with the present embodiment is the same as the unit searchinformation (153A and 154A) formed in accordance with the firstembodiment, except that the elements forming the unit search informationare changed from the search information elements (153,154) themselves torecord position information (153D, 154D).

Therefore, the types, method of extraction and the like of the sharedinformation to be handled similar to those described in the firstembodiment are also applied to the present embodiment. Thus, effectssimilar to those obtained by the first embodiment are also obtained inthe present embodiment.

Unit search information control unit 105A is connected to a processingunit 16, and, by way of example, when processing unit 16 is atransmitting unit, corresponding unit search information is transmitted,in response to a request from a destination of transmission. Unit searchinformation control unit 105A in accordance with the present embodimentreceives a request based on record position information 153D or 154D ofthe semantic or signal search information element 153 or 154 included inalready transmitted unit search information, draws out the semantic orsignal search information element 153 or 154 that is indicated by therecord position information 153D or 154D corresponding to the request,and transmits the same to the source of request.

Though not described in the present embodiment, for various pieces ofstructural information 50 drawn out from structural search informationelement 152 to be used, not the data themselves of structuralinformation 50 but information representing record position may beextracted similar to that for the above described semantic searchinformation element 153 and signal search information element 154, andthe unit search information may be formed using the extracted recordposition information.

In this case, from the record position information of structuralinformation 50 and the record position information 153D of semanticsearch information element 153, a semantic unit search information isformed, and from the record position information of structuralinformation 50 and the record position information 154D of the signalsearch information element 154, a signal unit search information isformed.

The schematic configuration of the search apparatus 2A in accordancewith the present embodiment is shown, similar to the first embodiment,in FIG. 7. The function of search apparatus 2A is slightly differentfrom that of the first embodiment, as will be described below.

Search apparatus 2A in accordance with the present embodiment includes asearch executing unit 701A, a semantic unit search information holdingunit 702A, a signal unit search information holding unit 703A, and asearch information control unit 704A. Semantic unit search informationholding unit 702A and signal unit search information holding unit 703Ahold units of search information 153E and 154E, respectively, and inaddition they have the function of temporarily holding for every onesearch execution or continuously holding for a period while searches areexecuted, each of search information elements 153 and 154 obtained basedon respective record position information 153D and 154D included in thealready obtained units of search information 153E and 154E,respectively.

Search executing unit 701A selects one or an arbitrary combination of aplurality of various search information elements held in semantic unitsearch information holding unit 702A and signal unit search informationholding unit 703A, obtains search information elements in accordancewith the selected combination, and executes a search for the movingimage contents using the obtained one or more search informationelements.

When the search information elements are held only temporarily in unitsearch information holding units 702A and 703A, that is, when theelements are discarded at every execution of a search, search executingunit 701A starts a search not from a request for search informationelements but from a request for unit search information to unit searchinformation holding units 702A and 703A, as will be described in thefollowing. For simplicity of description, here, a search will bedescribed as an example, which starts with a request for semantic unitsearch information 153E.

The operation is the same when signal unit search information 154E isrequested or when both semantic unit search information 153E and signalunit search information 154E are requested.

When the selected semantic search information element 153 is not held inthe corresponding semantic unit search information holding unit 702A,search executing unit 70 1A requests corresponding unit searchinformation 153E from semantic unit search information holding unit702A.

When the requested unit search information 153E can be obtained fromcorresponding semantic unit search information holding unit 702A, searchexecuting unit 701A applies to search information control unit 704A, theobtained unit search information 153E together with a request for asearch information element. Based on the record position information153D included in the applied unit search information 153E, searchinformation control unit 704A obtains the corresponding semantic searchinformation element 153 from the transmitting unit or storing unit 19,and stores the obtained semantic search information element 153 incorresponding semantic unit search information holding unit 702A. Searchexecuting unit 701A executes a search using semantic search informationelement 153 obtained in this manner.

When the requested unit search information 153E is not held in thecorresponding semantic unit search information holding unit 702A andhence cannot be obtained, search executing unit 701A first issues arequest for the unit search information 153E to search informationcontrol unit 704A. Receiving the request as an input, search informationcontrol unit 704A obtains the corresponding unit search information 153Ein response to the input request from the transmitting unit or storingunit 19, and stores the obtained unit search information 153E in thecorresponding semantic unit search information holding unit 702A.Thereafter, search executing unit 701A applies, together with a requestfor a search information element, the obtained unit search information153E to search information control unit 704A. Based on the recordposition information 153D included in the applied unit searchinformation 153E, search information control unit 704A obtains thecorresponding semantic search information element 153 from thetransmitting unit or storing unit 19, and stores the obtained semanticsearch information element 153 in the corresponding unit searchinformation holding unit 702A. Search executing unit 701A executes asearch for moving. image contents (data of moving image contentsprepared by the transmitting unit or storing unit 19), using thesemantic search information element 153 obtained in this manner.

As will be understood from the foregoing, search apparatus 2A inaccordance with the present embodiment differs from search apparatus 2in accordance with the first embodiment in that, as regards the mannerof obtaining the search information, two steps are taken. Namely, therecord position information of the search information element isobtained from the unit search information, and from the obtained recordposition information, the corresponding search information element isobtained. Except for this point, the processes are the same. Therefore,similar effects as obtained in the first embodiment can also be obtainedin the present embodiment.

According to the present embodiment that uses unit search informationconsisting only of the record position information, only a small amountof signals have to be recorded, as compared with the unit searchinformation consisting of various search information elementsthemselves. Therefore, in such a situation that a searcher conducts asearch using a client terminal such as a portable information terminalhaving a small storage capacity, the search method is effective, inwhich only the unit search information in accordance with the presentembodiment is continuously held in the client terminal and only thenecessary search information elements are obtained from a serverapparatus (corresponding to the transmitting unit or storing unit 19 ofFIG. 7) using the unit search information at the time of executing asearch. The sever apparatus has data of moving image contents and dataof search information 15 that are separated into search informationelements in advance recorded therein, and the server has a function ofreading and outputting to the source of request the data of the movingimage contents or the data of search information 15 in response to arequest from the outside.

Therefore, it is possible to know, in a simple manner, what searchinformation exists in the server apparatus from the unit searchinformation, and efficiency of executing a search can be improved asonly the necessary search information is transmitted.

In the foregoing, the timing at which the unit search information isobtained from the transmitting unit or storing unit 19 has beendescribed as dependent on the request of the searcher. The descriptionis not limiting, and, by way of example, various units of searchinformation may only be automatically obtained by a client terminal atthe start of a search. There is no limit on the timing of obtaining ordiscarding the unit search information.

In the foregoing, holding time of the search information elementobtained based on the record position information in the unit searchinformation has been described as temporal, that is, the element isdiscarded every time a search is executed, or continuous, that is, theelement is held for the period of execution of searches. Thedescription, however, is not limiting, and such a management may bepossible in that the element is held for a prescribed time period anddiscarded if there is no access to the element in the prescribed timeperiod. There is no limit as to the holding time of the searchinformation element.

Though a search is executed based on the formed unit search informationin the search apparatus in accordance with the present embodiment, thesearch information of the original tree structure (part thereof) may berecovered from the obtained unit search information, and based on thesearch information of the recovered tree structure, a search may beexecuted, dependent on the manner or search. As each of the units ofsearch information formed by the search information managing apparatusin accordance with the present invention includes the structuralinformation, it is possible to recover the original tree structure byusing the structural information (structuring information).Specifically, by finding the position information in the tree structurefrom the structural information included in the unit search information,obtaining the corresponding search information element indicated by thecorresponding record position information in the unit searchinformation, and connecting the same as a leaf RF to the position of thetree structure indicated by the found position information, the treestructure is recovered. The recovery from the unit search information tothe tree structure can be used not only for the search but also fortransmission of the search information to other person through a narrownetwork, by dividing the tree into small pieces, transmitting each ofthe divided pieces to the destination (other person) and recovering thetree at the receiving side (other person) to be managed as the searchinformation.

Third Embodiment

FIG. 9 shows a schematic configuration of a search information managingapparatus 1B in accordance with the present embodiment. Searchinformation managing apparatus 1B includes a division selecting unit901, first search information dividing unit 902, second searchinformation dividing unit 903 and a unit search information control unit904. Division selecting unit 901 designates manner of search, that is,whether the desired manner of search for the moving image contents is anon-line search or an off-line search. In accordance with the designatedmanner of executing a search, the method of division of searchinformation 15, that is, the form of the unit search informationobtained by the division, is selected. Search information 15 is, forexample, the search information 15 having the tree structure shown inFIG. 2.

When an off-line search is designated and the unit search informationassociated with the types of search information elements is selected asthe method of dividing search information 15, division selecting unit901 instructs first search information dividing unit 902 to executedivision of search information 15. When an on-line search is designatedand unit search information consisting of time sections is selected asthe method of dividing search information 15, division selecting unit901 instructs second search information dividing unit 903 to executedivision of search information 15. In a special example in which anoff-line search and an on-line search are conducted simultaneously,division selecting unit 901 instructs both the first and second searchinformation dividing units 902 and 903 to execute division of searchinformation 15 (formation of unit search information).

First search information dividing unit 902 receives as an input thesearch information 15, executes division of search information 15associated with the types of search information elements as described inthe first (second) embodiment, and forms and outputs units of searchinformation 153A (153E) and 154A (154E) associated with the types ofsearch information such as shown in FIGS. 4A to 4C and 5A to 5C (8A to8D).

Second search information dividing unit 903 receives as an input thesearch information 15, executes division using time section as a unit asshown in FIGS. 13A to 13D or FIG. 14, and forms and outputs unit searchinformation along time section.

Unit search information control unit 904 is connected to processing unit16, selects one or both of the unit search information associated withthe types of search information elements formed by first searchinformation dividing unit 902 and the unit search information along thetime section formed by second search information dividing unit 903 inresponse to a request from processing unit 16, and outputs the selectedinformation to the source of request.

As the division selecting unit 901 is provided, appropriate control ofsearch information (formation of unit search information) in accordancewith the manner of executing a search for the moving image contentsbecomes possible.

The invention claimed is:
 1. A search information managing apparatusmanaging search information, stored on a tangible machine-readablemedium, having a tree structure corresponding to moving image contents,wherein: said search information contains a plurality of searchinformation elements of a same type attached to a same hierarchicallevel of the tree structure, wherein each of said plurality of searchinformation elements is given by a leaf of the tree structure or givenby a sub-tree of the tree structure consisting of a node and anarbitrary set of leaves or nodes of lower levels linked to the node, andcontains structural information corresponding to said plurality ofsearch information elements; said apparatus comprising: a structuralinformation compressor that removes redundant information from saidstructural information corresponding to said plurality of searchinformation elements whereby to reduce a total quantity of saidstructural information; and a unit search information formation sectionthat forms a unit of search information by arranging said plurality ofsearch information elements only of the same type and corresponding saidstructural information compressed by said structural informationcompressor in accordance with a prescribed format, wherein saidstructural information compressor separates said structural informationinto shared structural information including type information shared bysaid plurality of search information elements or shared positioninformation of said tree structure, and un-shared structural informationother than said shared structural information; and said unit searchinformation formation section forms a unit of search information byarranging, in addition to said arrangement of said plurality of searchinformation elements, said shared structural information and saidun-shared structural information.
 2. The search information managingapparatus according to claim 1, wherein said plurality of searchinformation elements formed into said unit of search information by saidunit search information formation section correspond to signal searchinformation representing physical characteristics of signals of saidmoving image contents.
 3. The search information managing apparatusaccording to claim 1, wherein said plurality of search informationelements formed into said unit of search information by said unit searchinformation formation section correspond to semantic search informationrepresenting characteristics in semantic context of said moving imagecontents.
 4. The search information managing apparatus according toclaim 1, wherein said prescribed format represents an arrangement ofsaid plurality of search information elements formed into said unit ofsearch information by said unit search information formation section inaccordance with an order of appearance of said plurality of searchinformation elements in said tree structure.
 5. A search informationmanaging apparatus managing a unit of search information stored on atangible machine-readable medium, said unit of search information formedfrom search information having a tree structure corresponding to movingimage contents, wherein said search information contains a plurality ofsearch information elements of a same type attached to a samehierarchical level of the tree structure, wherein each of said pluralityof search information elements is given by a leaf of the tree structureor given by a sub-tree of the tree structure consisting of a node and anarbitrary set of leaves or nodes of lower levels linked to the node, andcontains structural information corresponding to said plurality ofsearch information elements; said apparatus comprising: a structuralinformation compressor that removes redundant information from saidstructural information corresponding to said plurality of searchinformation elements whereby to reduce a total quantity of saidstructural information; and a unit search information formation sectionthat forms a unit of search information by arranging (i) record positioninformation corresponding to each of said plurality of searchinformation elements only of the same type, representing a positionwhere the corresponding search information element is recorded, and (ii)said structural information compressed by said structural informationcompressor, in accordance with a prescribed format, wherein saidstructural information compressor separates said structural informationinto shared structural information including type information shared bysaid plurality of search information elements or shared positioninformation of said tree structure, and un-shared structural informationother than said shared structural information; and said unit searchinformation formation section forms said unit of search information byarranging, in addition to said arrangement of said record positioninformation, said shared structural information and said un-sharedstructural information.
 6. The search information managing apparatusaccording to claim 5, wherein said plurality of search informationelements correspond to signal search information representing physicalcharacteristics of signals of said moving image contents.
 7. The searchinformation managing apparatus according to claim 5, wherein saidplurality of search information elements correspond to semantic searchinformation representing characteristics in semantic context of saidmoving image contents.
 8. The search information managing apparatusaccording to claim 5, wherein said prescribed format represents anarrangement of said record position information in accordance with anorder of appearance of search information elements corresponding to saidrecord position information in said tree structure.
 9. A searchinformation managing method managing search information stored on atangible machine-readable medium, having a tree structure correspondingto moving image contents, wherein: said search information contains aplurality of search information elements of a same type attached to asame hierarchical level of the tree structure, wherein each of saidplurality of search information elements is given by a leaf of the treestructure or given by a sub-tree of the tree structure consisting of anode and an arbitrary set of leaves or nodes of lower levels linked tothe node, and contains structural information corresponding to saidplurality of search information elements; said method comprising thefollowing steps: a structural information compressing step of removingredundant information from said structural information corresponding tosaid plurality of search information elements whereby to reduce a totalquantity of said structural information; and a unit search informationforming step of forming a unit of search information by arranging saidplurality of search information elements only of the same type andcorresponding said structural information compressed by said structuralinformation compressing step in accordance with a prescribed format,wherein said structural information compressing step separates saidstructural information into shared structural information including typeinformation shared by said plurality of search information elements orshared position information of said tree structure, and un-sharedstructural information other than said shared structural information;and said unit search information forming step forms a unit of searchinformation by arranging, in addition to said arrangement of saidplurality of search information elements, said shared structuralinformation and said un-shared structural information.
 10. A searchinformation managing method managing units of search information storedon a tangible machine-readable medium, said units of search informationformed from search information having a tree structure corresponding tomoving image contents, wherein: said search information contains aplurality of search information elements of a same type attached to asame hierarchical level of the tree structure, wherein each of saidplurality of search information elements is given by a leaf of the treestructure or given by a sub-tree of the tree structure consisting of anode and an arbitrary set of leaves or nodes of lower levels linked tothe node and contains structural information corresponding to saidplurality of search information elements; said method comprising thefollowing steps: a structural information compressing step of removingredundant information from said structural information corresponding tosaid plurality of search information elements whereby to reduce a totalquantity of said structural information; and a unit search informationforming step of forming a unit of search information by arranging (i)record position information corresponding to each of said plurality ofsearch information elements of only the same type, representing aposition where the corresponding search information element is recorded,and (ii) said structural information compressed by using said structuralinformation compressing step, in accordance with a prescribed format,wherein said structural information compressing step separates saidstructural information into shared structural information including typeinformation shared by said plurality of search information elements orshared position information of said tree structure, and un-sharedstructural information other than said shared structural information;and said unit search information forming step forms a unit of searchinformation by arranging, in addition to said arrangement of said recordposition information corresponding to each of said plurality of searchinformation elements only of the same type, said shared structuralinformation and said un-shared structural information.
 11. A searchapparatus executing a search based on a unit of search informationformed from search information stored on a tangible machine-readablemedium, said search information having a tree structure corresponding tomoving image contents, wherein said unit of search information is searchinformation formed by arranging, for each of a plurality of searchinformation elements given by a plurality of leaves of the treestructure or given by a plurality of sub-trees consisting of anarbitrary set of a node and leaves or nodes of lower levels linked tothe node of the tree structure representing elements only of a same typeattached to a same hierarchical level of the tree structure included insaid search information, record position information representing aposition where the search information element is recorded andcorresponding structural information compressed from said searchinformation and having redundant information removed from saidstructural information whereby to reduce a total quantity of saidextracted structural information, in a prescribed format, wherein saidstructural information is compressed so as to separate said structuralinformation into shared structural information including typeinformation shared by said plurality of search information elements orshared position information of said tree structure, and un-sharedstructural information other than said shared structural information;and said unit search information forms a unit of search information byarranging, in addition to said arrangement of said record positioninformation the positions where each of said plurality of searchinformation elements are recorded, said shared structural informationand said un-shared structural information; said search apparatuscomprising: first holding means for holding said unit of searchinformation; second holding means for holding said plurality of searchinformation elements; and search executing means for executing a searchof said moving image contents using said unit of search information andsaid plurality of search information elements.
 12. A search informationmanaging apparatus for recovering a part of search information from aunit of search information stored on a tangible machine-readable medium,said unit of search information being formed from search informationhaving a tree structure corresponding to moving image contents, whereinsaid unit of search information is at least one of units of searchinformation formed of semantic search information representingcharacteristics in semantic context of the moving image contents orsignal search information representing physical characteristics ofsignals of the moving image contents, and includes structuralinformation obtained by disassembling structural search informationrepresenting the structure of the moving image contents and removingredundant information from said structural information whereby to reducea total quantity of said structural information, wherein the unit ofsearch information is formed of only same type of search informationelement; said apparatus comprising: a position calculator forcalculating, from said structural information included in said unit ofsearch information, a position in said tree structure that correspondsto each of said semantic search information or said signal searchinformation included in said unit of search information; and arecovering section that recovers a part of said search information, byconnecting, to the position in said tree structure calculated by saidposition calculator, corresponding said semantic search information orsaid signal search information, wherein said structural informationconsists of shared structural information including type informationshared by said plurality of search information elements or sharedposition information of said tree structure, and un-shared structuralinformation other than said shared structural information; and said unitof search information is formed by arranging, in addition to saidarrangement of said plurality of search information elements, saidshared structural information and said unshared structural information.13. A search information managing apparatus for recovering a part ofsearch information from a unit of search information stored on atangible machine-readable medium, said unit of search information havingbeen obtained by dividing search information having a tree structurecorresponding to moving image contents, wherein said unit of searchinformation is formed by arranging, in a prescribed format, a pluralityof search information elements only of a same type attached to a samehierarchical level of the tree structure included in said searchinformation, wherein each of said plurality of search informationelements is given by a leaf of the tree structure or given by a sub-treeof the tree structure consisting of a node and an arbitrary set ofleaves or nodes of lower levels linked to the node, and structuralinformation corresponding to said plurality of search informationelements being compressed wherein redundant information has been removedfrom structural information originally contained in said searchinformation whereby to reduce a total quantity of said structuralinformation; said apparatus comprising: a position calculator forcalculating, from said structural information included in said unit ofsearch information, a position in said tree structure that correspondsto each of said plurality of search information elements included insaid unit of search information; and a recovery section that recovers apart of said search information, by connecting, to the position in saidtree structure calculated by said position calculator, each of saidcorresponding plurality of search information elements as a leaf or asub-tree, wherein said structural information consists of sharedstructural information including type information shared by saidplurality of search information elements or shared position informationof said tree structure, and un-shared structural information other thansaid shared structural information; and said unit of search informationis formed by arranging, in addition to said arrangement of saidplurality of search information elements, said shared structuralinformation and said unshared structural information.
 14. A searchinformation managing apparatus for recovering a part of said searchinformation from a unit of search information stored on a tangiblemachine-readable medium, said unit of search information formed fromsaid search information having a tree structure corresponding to movingimage contents, wherein said unit of search information is formed byarranging, for each of a plurality of search information elements onlyof a same type attached to a same hierarchical level of the treestructure included in said search information, record positioninformation indicating a position where the corresponding searchinformation element is recorded, wherein each of said plurality ofsearch information elements is given by a leaf of the tree structure orgiven by a sub-tree of the tree structure consisting of a node and anarbitrary set of leaves or nodes of lower levels linked to the node, andthe structural information corresponding to said plurality of searchinformation elements being extracted wherein redundant information hasbeen removed from structural information originally contained in saidsearch information whereby to reduce a total quantity of structuralinformation; said apparatus comprising: a position calculator forcalculating, from said structural information included in said unit ofsearch information, a position in said tree structure corresponding toeach of said plurality of search information elements indicated by saidrecord position information included in said unit of search information;a search information element acquisition section that obtains acorresponding plurality of search information elements based on saidrecord position information included in said unit of search information;and a recovery section that recovers a part of said search informationby connecting, to the position in said tree structure calculated by saidposition calculator, said corresponding plurality of search informationelements obtained by said search information element acquisition sectionwherein said structural information consists of shared structuralinformation including type information shared by said plurality ofsearch information elements or shared position information of said treestructure, and unshared structural information other than said sharedstructural information; and said unit of search information is formed byarranging, in addition to said arrangement of said record positioninformation, said shared structural information and said un-sharedstructural information.
 15. The search information managing apparatusaccording to claim 1, wherein said structural information includesinformation representing an original position of said plurality ofsearch information elements in said tree structure of said searchinformation.
 16. The search information managing apparatus according toclaim 1, wherein said structural information includes informationrepresenting a time section of said moving image contents, and said timesection of said moving image contents corresponds to said plurality ofsearch information elements.
 17. The search information managingapparatus according to claim 5, wherein said structural informationincludes information representing position of said plurality of searchinformation elements in said tree structure of said search information.18. The search information managing apparatus according to claim 5,wherein said structural information includes information representing atime section of said moving image contents, and said time section ofsaid moving image contents corresponds to said plurality of searchinformation elements.
 19. The search information managing apparatusaccording to claim 13, wherein said prescribed format represents saidplurality of search information elements included in said unit of searchinformation arranged in accordance with an order of appearance thereofin said tree structure.
 20. The search information managing apparatusaccording to claim 14, wherein said prescribed format represents saidrecord position information included in said unit of search informationrespectively arranged in accordance with an order of appearance of saidplurality of search information elements corresponding to said recordposition information in said tree structure.
 21. The search informationmanaging apparatus according to claim 1, wherein a single unit of searchinformation is formed from the plurality of search information element.